A Subsampling Line-Search Method with Second-Order Results

In many contemporary optimization problems such as those arising in machine learning, it can be computationally challenging or even infeasible to evaluate an entire function its derivatives. This motivates the use of stochastic algorithms that sample problem data, which jeopardize guarantees obtained through classical globalization techniques optimization, a line search. Using subsampled values is particularly for latter strategy, relies upon multiple evaluations. For nonconvex data-related problems, training deep learning models, one aims at developing methods converge second-order stationary points quickly, is, escape saddle efficiently. difficult ensure when only accesses approximations objective and this paper, we describe algorithm based on negative curvature Newton-type directions are computed subsampling model objective. A line-search technique used enforce suitable decrease model; sufficiently large sample, similar amount reduction holds true We then present worst-case complexity notion stationarity tailored context. Our analysis encompasses deterministic regime allows us identify sampling requirements paradigms. As illustrate real data experiments, these estimates need not satisfied our method competitive with first-order strategies practice.